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An  Upper  Bound  for  the  Number  of  Certain 
Error  Correcting  Codes 


Let  S  "be  the  set  of  n  dimensional  vectors  whose  components 
may  take  only  values  0  or  1-  An  error  correcting  code  is  a  sulset 
T  of  S  having  the  property  that  any  pair  of  vectors  in  T  differ  in 
at  least  d  components.  The  general  question  of  how  numerous  a  set 
T  may  he  found  for  given  n  and  d  is  of  importance  in  information- 
theory  and  will  he  treated  in  this  discussion. 

Fano    has  shown  that  if  a  set  T  can  he  found  having  the 
property  that  every  member  of  S  lies  a  distance  of  less  than  d/2 

from  some  member  of  set  T,  then  the  number  of  members  of  the  set 

i 
T  approaches  2 

"by  the  formula 


nK 
T  approaches  2   as  n  approaches  infinity.  The  quantity  K  is  given 


K  =  1  +  (log  d/2n)  d/2n  +  (log  (l  -  d/2n))(l  -  d/2n) 


(1) 


nK 
This  number  2   represents  an  upper  hound  for  the  numbesr  of  members 

of  the  set  T,and  if  it  could  he  achieved  the  resulting  set  T  would 

he  an  error  correcting  code  of  maximum  efficiency. 

Fano's  formula  has  not  been  achieved  by  actual  codes  in 

which  d/2n  is  held  constant  as  n  -»  OO   ,   and  there  may  even  be 

some  doubt  as  to  whether  any  non-zero  constant  K  may  be  found  to 

satisfy  the  formula  2   .  In  the  special  case  of  n  =  2?   where  p  is 

an  integer  it  is  possible  to  construct  a  set  T  whenever  d  is  a  power 

of  2 ,  say  2  .  The  number  of  members  of  this  set  T  is  given  "by  the 


formula 


EI*].  (2) 

i=0 


The  sum  of  binomial  coefficients  in  the  exponent  will  be  a  polynomial 

p 
in  p  of  degree  p-m  and  consequently  cannot  be  proportional  to  n  =  2  . 

Robert  M.  Fano  "Notes  for  Statistical  Theory  of  Information"  (1953) 
Page  IV- 3k,     Mass.  Inst,  of  Technology 
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In  the  case  m  ■  p-1  we  get  d  =  2P"     =  n/2  and  the  number  of  Members 
of  set  T  is  merely  2  p  -  2n.  The  theorem  given  here  treats  the  case 
of  general  n  with  d  =  n/2  and  shows  that  the  set  T  can  centain  no 
more  than  2n  members  and  that  the  formula  of  Fano  gives,  therefore,  a 
much  higher  than  necessary  upper  bound  in  this  ease. 

Let  us  designate  the  number  of  components  in  which  two  vecters 
a  and  b  differ  by  the  symbol  L(a,b).  The  condition  that  d  =  n/2  may 
then  be  written  as  L(a,b)  >  n/2  if  a  and  b  are  in  T« 

THEOREM:   It  is  not  possible  to  pick  a  subset  T  of  more  than 
2n  members  from  the  set  S  of  n  dimensional  vectors  whose  components  are 
0  or  1  if  one  requires  that  any  pair  a,  b  from  set  T  satisfy  L(a,b)  >  n/2. 
If  the  condition  is  made  L(a,b)  >  n/2  it  is  never  possible  to  pick 
more  than  n+1. 

PROOF t    Let  (aQ,  a.,...,  a  ,)  represent  the  n  component  vector 
"aM  whose  components  a.  are  either  0  or  1.  Let  us  adopt  the  notation 
(A- ,  A_ , . . .  ,A  ,  )  to  represent  the  transformed  vector  A,  where  the  vector 
A  will  be  transformed  by  the  formula 

k±  -  (l-aa^/fi  (1) 

Thus  if  a.  =  0,  A  =  l/  fn  and  if  a  ■  1,  A  =  -l/  fn".  These  are  the 
only  two  possible  choices  for  A.  and  consequently 

n-1     o 

z    (A.r  =  i.  (2) 

i=0    x 

We  shall  let  V  represent  the  broader  class  of  vectors  satisfying  (2 ) 
(these  are  unit  vectors).  Transforms  from  set  S  are  in  V  but,  of  course, 
V  contains  vectors  which  are  not  transforms  from  S.  The  theorem  will  be 
proven  for  set  V  and  as  a  result  will  apply  to  the  smaller  Set  S. 

Formula  (l)  allows  a  numerical  representation  of  the  quality 
L(a,b).  If  A.  is  the  transform  of  a.  and  B.  is  the  transform  of  h 
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then  l/2  -  n/2  A.  B.  will  be  0  If  a  and  b  agree  and  1  if  a.  and  b 
disagree.  By  summing  such  expressions  we  obtain 

n-1 
L(a,b)  =  (n/2)  (l  -  2   A,  B  ) .  (3) 

1=0 

This  formula  permits  the  condition  L(a,b)  >  n/2  to  be  written 

n-1  n-1 

Z   A .  B  <  0  and  the  condition  L(a,b)  >  n/2  to  be  written  Z  A.B  <  0. 

1=0       ""  1=0  1  1 

Let  us  assume  that  a  set  T  has  been  found  containing  e  members 

r  ,r  , ...,r  ~  which  satisfy  the  condition  L(r  ,r  )  >  n/2  if  j  4  k» 

The  transformed  set  V  contains  e  members  R  ,  R  ,  . . . ,  R    and  satisfies 

the  condition 

n-1   .  , 

Z   R^  R?   CO  (k) 

1=0 

The  set  of  vectors  V  may  be  rotated  if  each  vector  is  post  multiplied 

by  a  unitary  matrix  M.  Under  such  a  rotation  the  scalar  product 

a-1 

Z   A.B   **  invariant  and  condition  (k)   will  therefore  continue  to  hold. 

1=0 

n-1 
To  show  that  Z  A  B  is  invariant  under  a  rotation  we  need  merely 

1=0  1  i 

write  Z    A.B.  =  AB  .   But  A'  =  AM  and  B'  =  BM.  Hence  A' (BT  )  = 
1=0 

rn        rp  rp.  .  rp      — "t 

AM  (BM)  =  AMM  B  '.  M,  however,  is  unitary  so  that  M"^  =  M   and 

AMM'V  =  AIBT  =  ABT. 

We  now  show  that  it  is  possible  to  rotate  V  into  V  so  that 
(R  )'  =  (1,0,0,0, .. .,0).  lit  other  words  we  shall  perform  a  rotation  M 
so  that  R  transforms  into  (1,0,0, .. .,0).  We  begin  by  performing  a 
rotation  by  M.  ,  where  M.  is  a  unitary  matrix  defined  as  follows, 
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,    "5 

< 

w  *  w 

i^f  *  («;)2 

< 

W  ♦  (»lf 

\|(»0°)2+(»?)2 

0 

0 

0 

0 

. ..   0 


...   0 

...   0 
...   0 


When  one  post  multiplies  R  by  M.  no  components  axe  changed  except  1L. 

■which  becomes  \(Rn )  +  (R,  )     and  R,  which  becomes  0 .  Similarly  a 

u      L  f4         0     0        0 

matrix  VL   may  be  constructed  which  alters  only  RQ  and  R_,  making  Rp 

equal  to  0  etc.  The  matrix  M  =  M_  ML  M,,...,M  ,  will  thus  rotate  R 

into  (1,0,0, ...,0). 

We  now  remove  (R  )'  from  Vf  and  place  it  in  a  set  W.  If  the 

•vector  (-1,0,0, .. .,0)  is  in  VT  we  also  place  it  in  W,  removing  it  from 

V  .  We  see  that  all  remaining  vectors  in  V  must  have  nonr^positive 

n"1    0     1      1 
first  components  since  Z   (R  )'  (R^)1  =  (R^)'  <  0.   Thus  we  may  write 

i=0    1     1 


the  condition 


n-1 

Z 

i=0 


Z  (rJ)«  (R*)<  <0 


whenever  (R  )'  and  (R  )'  are  in  V  as 


n-1 


(r£)»  (Rq)'  +Z  (R^)«  (R*)«  <0  . 
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But  since  (R^)'  s-nd  (R^V  ore  both  non~poaltiv»  we  have  (Uj*)'  (R^)'  >  0 

n-1    .     . 
and  hence  Z       (X0)1    (R  )'  <  0  (5) 

1=1    l     1      " 

Condition  (5)  say  now  replace  condition  (fc)  with  one  less  dimension 

1 

and  with  one  or  two  less  vectors  In  V   than  we  had  in  V.  It  is  tr^e  that 
the  normalization  condition  (2 )  may  not  hold  for  the  n-1  dimensions  hut 
condition  (5)  will  not  he  affected  if  we  multiply  any  vectors  in  V  hy 
suitable  constants  to  make  it  hold.  We  see  that  any  vector  in  V  whose 
last  n-1  components  are  zero  has  been  removed  from  V  and  placed  in  W 
so  that  normalization  is  always  possible. 

The  entire  process  described  here  may  be  repeated  for  the  set  V* 
in  n-1  dimensions  and  a  new  set  V"  constructed.  Bach  time  this  process 
is  repeated  we  put  at  most  two  vectors  in  W  and  lose  one  dimension.  Thus, 
after  n  repetitions  we  have  placed  all  e  vectors  in  W  and  hence  ©  may  be 
not  greater  than  2n.  This  proves  the  first  half  of  the  theorem. 

To  prove  the  second  part  of  the  theorem  we  follow  the  same  process 
but  use  the  condition 

E   R^  R*<0.  (6) 

1=0 

Using  this  condition  we  see  that  if  both  the  vectors  (l,  0,  0,...,0)  and 
(-1,0,0, ...,0)  are  placed  in  W  then  no  other  vector  may  be  left  in  V  and 
satisfy  (6 ) .  Thus  only  one  vector  is  placed  in  W  at  each  step  prior  to 
the  last  one  since  to  place  two  vectors  in  W  would  cut  short  the  process 
(make e  less  than  necessary).  At  the  last  step  two  vectors  may  be  placed 
in  W  since  no  more  vectors  will  be  left  In  V.  Hence  e  may  not  be  greater 
than  n+1  in  this  case  and  the  second  half  of  the  theorem  is  proved. 


( 


